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AIRWAY  SCIENCE  CURRICULUM  DEMONSTRATION  PROJECT: 


SUMMARY  OF  INITIAL  EVALUATION  FINDINGS 


INTRODUCTION 

The  Airway  Science  Curriculum  Demonstration  Project  was  designed  to 
investigate  the  effectiveness  of  an  alternative  approach  to  recruitment  and 
selection  within  five  of  the  Federal  Aviation  Administration's  (FAA)  major 
occupations.  Traditionally,  individuals  with  highly  technical  backgrounds 
have  been  hired  into  the  fields  of  Aviation  Safety  Inspector,  Electronics 
Technician,  and  Computer  Specialist.  In  addition,  prior  to  1981,  many  of  the 
incoming  Air  Traffic  Control  Specialists  had  former  military  experience  in 
controlling  air  traffic.  Since  that  time,  however,  proportional ly  fewer 
entrants  have  been  found  to  have  air  traffic  control  experience;  their 
training,  once  hired  into  the  FAA,  is  technical. 

The  designers  of  the  Airway  Science  program  believed  that  the  FAA's  focus  on 
technical  skills  and  expertise  might  limit  employees'  ability  to  smoothly 
adjust  to  the  substantial  technological  and  other  changes  that  they  are 
beginning  to  experience.  Also,  it  was  felt  that  the  strong  technical 
orientation  might  not  serve  employees  as  well  as  a  broader  background  as  they 
rise  to  supervisory  and  managerial  positions.  The  Airway  Science  project 
reflects  the  FAA's  interest  in  the  implications  of  complementing  the  changing 
technological  and  managerial  demands  within  the  FAA  with  a  professional 
workforce  whose  background  combines  technical  knowledge  with  exposure  to 
subjects  ranging  from  aviation,  the  physical  sciences,  and  computer 
technology  to  management,  the  social  sciences,  and  the  humanities. 

In  coordination  with  the  FAA,  a  baccalaureate  curriculum  was  developed  by  the 
University  Aviation  Association  (UAA),  a  professional  organization  of 
non-engineering  collegiate  aviation  educators.  The  resulting  Airway  Science 
degree  includes  core  coursework  in  general  studies,  sci ence/techrol ogy / 
mathematics,  management,  computer  science,  and  aviation.  In  addition,  five 
areas  of  concentration  (AOC)  supplement  this  curriculum  core:  Airway  Science 
Management,  Aircraft  Systems  Management,  Airway  Computer  Science,  Aviation 
Maintenance  Management,  and  Airway  Electronic  Systems.  Academic  institutions 
interested  in  receiving  official  FAA  recognition  of  their  Airway  Science 
programs  undergo  a  thorough  review  which  includes  a  detailed  examination  of 
the  proposed  curriculum  as  well  as  a  visit  to  the  school  to  determine  whether 
the  program  meets  the  Airway  Science  program  spirit  and  intent.  The  FAA  has 
contracted  with  the  UAA  for  the  services  of  the  UAA  Airway  Science  Curriculum 
Committee  to  conduct  this  program  review  function  and  to  make  a 
recommendation  to  the  FAA  regarding  the  merit  of  each  Airway  Science 
curriculum  proposal . 

Under  the  auspices  of  the  Demonstration  Project,  the  FAA  was  authorized  to  ed  J 

create  an  Airway  Science  register  to  facilitate  the  hiring  of  individuals  tion - 

with  the  previously  described  background.  It  should  be  noted  that  the  - - 

relevant  rating  guides  permit  comparable  work  experience  or  training  to  ________ 

substitute  for  educational  requirements.  Therefore,  the  Demonstration  i0n/ 
Project  includes  individuals  who  qualify  on  the  basis  of  education  alone,  lltv~T~T 
experience  alone,  or  some  education/experience  combination.  Airway  Science  -  — 

candidates  are  eligible  for  the  following  FAA  occupations:  Air  Traffic  and'r°r 

-  .  -✓ecial 

i  II. 


Control  Specialist  (ATCS)  (6S-2152);  Aviation  Safety  Inspector  -  Operations 
(ASI-OP)  (GS-1825);  Aviation  Safety  Inspector  -  Airworthiness  (ASI-AW) 
(GS-1825) ;  Computer  Specialist  (CS)  (GS-334);  and  Electronics  Technician  (ET) 
(GS-856) . 

The  Demonstration  Project  evaluation  design  was  discussed  In  detail  In  the 
report,  "An  Interim  Report  on  the  Airway  Science  Curriculum  Demonstration 
Project:  Clarification  and  Elaboration  of  the  Project  Design"  (Clough, 
1986b).  As  the  report  explained,  two  separate  research  questions  are  meshed 
within  the  Airway  Science  Curriculum  Demonstration  Project  objectives.  The 
original  project  design  was  based  on  the  premise  that  Individuals  with  an 
Airway  Science  background  were  not  being  hired  into  the  relevant  FAA 
occupations  through  traditional  methods.  As  a  result  of  this  assumption,  an 
alternative  selection  strategy  was  developed  which  abandoned  many  of  the 
traditional  selection  criteria  In  favor  of  evidence  of  an  Airway  Science 
background.  Thus,  new  rating  guides  were  developed  for  each  occupation  which 
attempted  to  identify  those  knowledges,  skills,  abilities*,  and  otiiei 
characteristics  (KSAOs)  that  completion  of  an  Airway  Science  degree  provided, 
and  then  to  equate  work  experience  and  college  education  with  respect  to 
these  KSAOs. 

The  current  study  compared  Airway  Science  hires  (those  hired  using  this 
alternative  selection  strategy)  with  traditional  hires  (those  hired  on  the 
basis  of  the  standard  selection  criteria).  Specifically,  the  experimental 
group  (ASC  HIRES)  comprised  Individuals  selected  from  an  Airway  Science 
register  Into  one  of  the  five  FAA  occupations.  The  FAA  Special  Examining 
Division  In  Oklahoma  City  determines  which  Airway  Science  applicants  are 
eligible  for  this  register  which  dictates  who  may  be  placed  In  the 
experimental  group.  ASC  HIRES  were  further  subdivided  on  the  basis  of 
program  eligibility  Into  a  5  x  3  matrix  which  considers  relevant  college 
education  along  one  axis  and  relevant  work  experience  along  the  other.  The 
comparison  group  (TRAD  HIRES)  contained  Individuals  hired  by  traditional 
selection  strategies  since  October  1983  who  agreed  to  participate  1o  the 
project  evaluation  by  signing  a  consent  form.  This  study  was  designed  to 
Investigate  whether  the  Airway  Science  selection  strategy  results  In:  (a) 
the  hiring  of  a  different  type  of  employee;  and  (b)  differences  In 
organizational  measures  of  job  satisfaction  and  success. 

A  second  study  was  added  to  the  demonstration  project  evaluation  to  examine 
more  closely  the  Influences  of  college  education  on  the  demonstration  project 
objectives.  Candidates  with  bachelor's  degrees  similar  in  some  respects  to 
the  Airway  Science  major  (as  well  as  Airway  Science  graduates  themselves) 
were  eligible  for  the  experimental  group  In  this  study.  The  method  by  which 
they  were  selected  (Airway  Science  or  traditional  strategy)  was  not  a  factor 
In  their  placement  Into  the  experimental  or  comparison  groups.  The  second 
study  and  findings  associated  with  analyzing  the  Information  In  this  manner 
were  described  In  detail  In  the  original  summatlve  evaluation  report  (Clough, 
1987).  In  general,  few  results  differed  from  those  found  In  comparing  ASC 
and  TRAD  HIRES.  One  reason  for  this  was  that  several  of  the  occupations  had 
few  individuals  who  qualified  for  the  experimental  group  on  the  basis  of  a 
specific  type  of  college  education.  Interested  readers  are  referred  to  the 
complete  agency  report;  the  second  study  is  not  further  discussed  in  this 
paper. 

This  Interim  report  summarizes  the  findings  of  the  Airway  Science  Curriculum 
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Demonstration  Project  during  Its  first  4  years.  The  agency  report  on  which 
this  paper  Is  based  (Clough,  1987)  provided  the  empirical  support  for  an  FAA 
request  to  the  Office  of  Personnel  Management  to  extend  the  duration  of  the  5 
year  demonstration  project  an  additional  4  years  (with  1  more  year  for 
evaluation  purposes).  Those  interested  in  more  specific  information  on  many 
facets  of  this  evaluation  effort  are  referred  to  the  complete  summatlve 
evaluation  report  (Clough,  1987). 

Organization  of  This  Report 

In  the  Method  section,  demographic  and  other  characteristics  of  the 
previously  described  project  evaluation  samples  are  presented.  In  addition, 
the  questionnaires,  supporting  databases,  and  data  collection  procedures  used 
to  execute  the  project  evaluation  are  briefly  reviewed.  The  accumulated 
evidence  for  each  of  thi  eight  project  objectives  Is  then  presented  In  the 
Results  section. 

The  Results  section  was  organized  on  the  basis  of  organizational  goals  which. 
In  turn,  subsume  specific  Airway  Science  project  objectives  (see  Appendix  A). 
An  Initial  attempt  was  made  to  combine  the  data  across  occupations  when 
occupation  per  se  was  not  of  central  Importance  to  the  project  objective. 
The  relative  Influences  of  hiring  status  and  occupation  could  then  be  tested. 
However,  at  times  It  appeared  that  there  were  real  differences  In  the 
response  patterns  of  Individuals  In  each  occupation.  When  empty  cells 
prevented  an  analysis  of  the  Interaction  between  hiring  status  and 
occupation,  this  statistical  approach  was  abandoned  and  examination  of  only 
the  ATCS  occupation  was  reported.  In  other  words,  when  it  was  not  possible 
to  compare  ASC  HIRES  and  TRAD  HIRES  within  each  occupation,  only  information 
relevant  to  the  ATCS  ASC  and  TRAD  HIRES  Is  presented.  All  statistical 
analyses  of  the  Items  In  the  Biographical  Questionnaire  Include  only  Air 
Traffic  Control  Specialists  (Termlnal/En  route)  for  It  was  the  only 
occupation  In  which  both  ASC  HIRES  and  TRAD  HIRES  responded. 

Two  occupational  groups  are  not  described  after  the  next  section  of  this 
report  although  both  are  currently  under  the  purview  of  the  Demonstration 
Project  evaluation.  The  occupations  are  (a)  Computer  Specialist  (GS-334)  in 
which  one  Airway  Science  candidate  had  been  hired  and  there  were  no 
comparison  group  members,  and  (b)  Air  Traffic  Control  Specialist  (Flight 
Service  Station)  In  which  one  Airway  Science  candidate  had  been  hired.  It 
was  inappropriate  to  even  include  these  people  In  overall  comparisons  between 
ASC  and  TRAD  HIRES  since  It  was  not  possible  to  determine  occupation-specific 
Influences.  [Note:  The  original  summatlve  evaluation  report  (Clough,  1987) 
stated  that  a  second  Computer  Specialist  had  been  selected;  the  Individual 
declined  the  appointment  following  the  completion  of  that  report  and 
therefore  was  not  Included  In  the  above  counts.] 

METHOD 


Project  Participants 

Air  Traffic  Control  Specialists  (Termlnal/En  route)  [ATCS  (T/E)].  As  of  May 
12,  1967,  96  ASC  HIRES  and  6774  TRAD  HIRES  had  entered  on  duty.  (Actually, 
according  to  our  records,  116  Airway  Science  candidates  had  been  selected; 
however,  all  were  not  yet  with  the  agency.)  Both  the  ASC  and  TRAD  HIRES 
samples  were  predominantly  men  (ASC:  83.7%;  TRAD:  85.7%)  and  nonminority 
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(ASC:  37.2%;  TRAD:  92.9%).  The  average  age  of  ASC  HIRES  (M  =  25.88)  and 

TRAD  HIRES  (M  =  25.83)  did  not  differ.  As  has  been  reported  in  the  past 

(Clough,  1986a;  1986c),  ASC  HIRES  were  much  more  likely  than  TRAD  HIRES  to  be 
certificated  as  a  ground  instructor,  flight  instructor,  and  pilot  (student, 
private,  commercial,  airline  transport).  Six  percent  of  the  ASC  HIRES  and 
11.6%  cf  the  TRAD  HIRES  had  worked  for  the  FAA  prior  to  their  appointments 
which  coincided  with  their  participation  in  the  Airway  Science  project.  The 
vast  majority  (79.5%)  of  ASC  HIRES  had  completed  a  Bachelor's  degree,  whereas 
only  29.7%  of  TRAD  HIRES  had  attained  that  educational  level.  Of  the  ATCS 
hires  who  had  attended  college,  significantly  more  ASC  than  TRAD  HIRES  had 
earned  credits  in  such  subject  areas  as  management,  computer  science,  and 
aviation. 

Respondents  wore  asked  to  classify  their  Associate  and  Bachelor's  degree 
aviation  coursework  into  discrete  categories.  For  example,  aviation 
coursework  taken  toward  an  Associate  and/or  Technical /Mi  1 itary/ 
Vocational-Technical  degree  was  described  as:  (a)  vocationally  oriented 

( i . e . ,  training  directed  toward  a  specific  occupation  and/or  FAA 
certif icate(s)  rather  than  toward  further  education  in  a  Bachelor's  degree 
program);  (b)  baccalaureate-transfer  oriented  (i.e.,  training  directed  Uw?rd 
further  education  in  a  Bachelor's  degree  program  rather  than  toward  a 
specific  occupation  or  FAA  certificate);  or,  (c)  other. 

Aviation  coursework  taken  toward  a  Bachelor's  degree  was  placed  into  one  of 
she  following  categories:  (a)  aviation  operations  (i.e.,  education  focusing 
th-  operation  of  aircraft  on  the  ground  or  in  the  air  such  as  Flight 
;";-ier,  Air  Traffic  Control,  or  Professional  Pilot);  (b)  aviation 
■  -JChr.ol  ogy  (i.e.,  education  focusing  on  ground  support  functions  such  as 
wionics  and  Aviation  Maintenance);  (c)  aviation  management  (i.e.,  education 
reusing  on  the  management  of  personnel  and/or  operations  or  systems  such  as 
Aviation  Administration/Management,  Air  Transportation  Management,  or  Airline 
.viement) . 

'ecus  of  the  aviation  coursework  taken  by  the  two  samples  differed  with 
proportionally  more  ASC  HIRES  concentrating  in  aviation  management;  the  vast 
majority  of  TRAD  HIRES  had  completed  coursework  focusing  on  aviation 
operations. 

'ISC  HIRES  were  classified  according  to  their  eligibility  for  the  Airway 
Soien.ro  register.  As  Table  1  illustrates,  60.0%  of  ASC  HIRES  qualified,  at 
least  in  part,  on  the  basis  of  an  Aviation  Bachelor's  degree.  In  addition, 
tvee  ">res  graduated  from  FAA  recognized  Airway  Science  programs  (two  of  the 
three  r.  :  not  yet  entered  on  duty).  Three  Aviation  graduates  had  prior  FAA 
expedience  as  Air  Traffic  Assistants  (GS-2154)  as  did  one  of  the  non-Aviation 
-■o j or? ,  in  addition,  one  Aviation  graduate  had  worked  with  the  FAA  in  a  non- 
ATj-related  position.  Six  Aviation  graduates  and  three  non-Aviation  majors 
entered  the  FAA  Academy  more  than  once;  it  is  not  known  whether  these 
enrollments  occurred  prior  to  or  following  their  Airway  Science  appointments. 

Electronics  Technicians  [ET] .  As  of  May  12,  1987,  48  ASC  and  22  TRAD  HIRES 
comprised  this  study.  (Our  records  indicated  50  selections,  with  48  having 
reported  for  duty.)  Most  ASC  HIRES  qualified  for  the  Airway  Science  register 
on  the  basis  of  a  combination  of  work  experience  and  college  education  (see 
Table  2).  All  ASC  HIRES  were  men  as  were  90.9%  of  the  TRAD  HIRES;  also,  the 
majority  of  both  samples  were  nonminority  (ASC:  88.6%;  TRAD:  72.7%).  The 
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Table  1.  Airway  Science  Eligibility  Ciassif ication  of  Air  Traffic  Control  Specialists  (Terminal/tn  route) 
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Ciassif ications  were  missing  for  16  Airway  Science  selections. 


Table  2.  Airway  Science  Eligibility  Classification 
(os  of  Moy  12,  1987) 

Relevant  Work  Experience 
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Classification  was  missing  for  one  ET  selection. 

Note.  ET  •  Electronics  Technician;  ASI-OP  -  Aviation  Safety  Inspector  -  Operations;  ASI-AW  -  Aviation  Safety 
Inspector  -  Airworthiness 


average  age  of  ASC  HIRES  (M  =  33.98)  and  TRAD  HIRES  (M  =  39.13)  did  not 
differ  significantly.  In  addition,  over  half  of  both  ASC  and  TRAD  HIRES 
received  veteran's  preference  (ASC:  65.2%;  TRAD:  75.0%).  Starting  grades 
of  TRAD  HIRES  ranged  from  GS-3  to  GS-12.  The  majority  of  ASC  HIRES  started 
in  the  specialty  option  of  Technical  Management  or  NAVAIDS/Communications 
whereas  TRAD  HIRES  tended  to  be  more  evenly  distributed  across  the 
specialties.  Some  change  in  specialties  has  been  observed  in  both  groups. 
Relatively  few  of  either  sample  had  earned  a  Bachelor's  degree  (ASC:  27.6%; 
TRAD:  7.7%),  although  an  additional  31.0%  of  ASC  HIRES  (and  38.5%  of  TRAD 
HIRES)  had  completed  an  Associate  degree. 

Aviation  Safety  Inspectors  z  Operations  [ASI-OP].  As  of  May  12,  1987,  data 
were  available  on  19  ASC  HIRES  and  82  TRAD  HIRES.  One  ASC  HIRE  had  an  Airway 
Science  degree,  42.1%  had  completed  an  Aviation  Bachelor's  degree  and  all  had 
prior  work  experience  (see  Table  2).  Four  of  the  19  ASC  HIRES  (21.1%)  were 
women,  whereas  only  three  of  the  82  TRAD  HIRES  (3.7%)  were  women.  Also, 
three  (15.8%)  ASC  HIRES  were  minority  (Hispanic);  only  5%  of  the  TRAD  HIRES 
sample  were  minority  (two  were  American  Indian,  one  Black,  and  one  Hispanic). 
The  average  age  of  ASC  HIRES  (M  =  33.42)  was  lower  than  that  of  TRAD  HIRES  (M 
=  42.86).  TRAD  HIRES  were  much  more  likely  than  ASC  HIRES  to  have  received 
veteran's  preference  (ASC:  38.9%;  TRAD:  81.8%).  Although  the  entire  ASC 
HIRES  sample  began  in  the  specialty  option  of  General  Aviation,  at  least  29% 
of  the  TRAD  HIRES  entered  within  the  specialty  of  Air  Carrier.  Two  ASC  HIRES 
have  since  converted  to  the  Air  Carrier  specialty.  Also,  all  ASC  HIRES  began 
as  GS-7s;  the  majority  of  TRAD  HIRES  entered  as  GS-lls  or  GS-12$  due 
presumably  to  their  extensive  flight  and  other  relevant  experience.  The  TRAD 
HIRES  sample  was  designed  to  resemble  (and  therefore  generalize  to)  the 
"typical"  person  hired  on  the  basis  of  traditional  hiring  criteria;  entrance 
at  the  GS-7  level  is  very  rare  in  this  occupation. 

Biographical  information  was  available  for  only  eight  ASC  HIRES  and  49  TRAD 
HIRES.  Of  those  responding,  six  ASC  HIRES  (75.0%)  and  22  TRAD  HIRES  (44.9%) 
had  earned  a  Bachelor's  degree.  An  additional  ten  (20.4%)  TRAD  HIRES  had 
completed  a  Master's  degree,  and  one  person  (2.0%)  reported  receiving  a 
Doctorate  or  professional  degree.  No  ASC  HIRES  had  earned  advanced  degrees. 
The  aviation  coursework  taken  by  the  two  samples  differed  consistently; 
proportionally  more  ASC  HIRES  took  Associate  degree  coursework  that  was 
Bachelor's  degree  oriented  and  Bachelor's  degree  coursework  concentrating  in 
aviation  operations. 

Aviation  Safety  Inspectors  z  Ai rworthiness  [ASI-AWj.  Ten  ASC  HIRES  and  115 
TRAD  HIRES  were  participating  in  this  study  as  of  May  12,  1987.  All  ASC 
HIRES  except  one  qualified,  at  least  in  part,  on  the  basis  of  prior  work 
experience  (see  Table  2).  Two  of  the  10  ASC  HIRES  (20%)  were  women,  whereas 
only  three  of  115  TRAD  HIRES  (2.6%)  were  women.  Similarly,  five  of  the  eight 
ASC  HIRES  (62.5%)  for  whom  minority  status  information  was  available  were 
nonminority;  in  contrast,  95.5%  TRAD  HIRES  were  nonminority.  The  average  age 
of  ASC  HIRES  (M  =  33.80)  was  considerably  lower  than  that  of  TRAD  HIRES  (M  = 
43.23).  Most  TRAD  HIRES  received  veteran's  preference  (92.0%);  however,  only 
30.0%  of  the  ASC  HIRES  were  veterans.  Biographical  information  was  available 
on  six  ASC  HIRES  and  78  TRAD  HIRES.  Of  those  responding,  two  ASC  HIRES 
(33.3%)  had  received  a  Bachelor's  degree,  one  (16.7%)  an  Associate  degree, 
and  the  remaining  three  (50.0%)  no  degree.  Of  the  TRAD  HIRES,  61.5%  had 
earned  no  college  degree,  21.8%  an  Associate,  15.4%  a  Bachelor's  degree,  and 
1.3%  an  advanced  degree.  The  types  of  aviation  coursework  did  not  differ 
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substantially  between  the  two  groups,  although  proportionally  more  ASC  HIRES 
took  more  aviation  credit  hours.  Tests  of  statistical  significance  could  not 
be  run  on  any  demographic  information  due  to  the  small  sample  sizes. 

Since  the  Airway  Science  register  permits  entry  at  the  GS-7  level  only,  all 
ASC  HIRES  began  at  that  grade;  on  the  other  hand,  over  80%  of  the  TRAD  HIRES 
started  as  GS- 11s  or  GS-12s  during  their  AS  I  appointment  which  coincided  with 
their  participation  in  this  project.  Once  again,  GS-7  is  a  lower  than  usual 
entry-level  grade  in  the  ASI  occupation.  The  TRAD  HIRES  sample  consists  of 
higher  grade  levels  because  it  was  designed  to  resemble  "typical"  TRAD  HIRES. 
Also,  all  ASC  HIRES  began  in  the  specialty  area  of  General  Aviation  or 
“Other,"  whereas  55.7%  of  TRAD  HIRES  began  in  the  Air  Carrier  specialty;  an 
additional  32.2%  of  the  sample  initially  were  in  General  Aviation.  Two  ASC 
HIRES  since  converted  to  the  Air  Carrier  specialty  area. 

The  remaining  occupations.  The  databases  for  the  remaining  occupations  ATCS 
(TTight  Service  Station  (FSS)]  and  Computer  Specialist  (CS),  contained 
relatively  little  information.  The  primary  reason  for  this  was  the  low 
number  of  Airway  Science  candidates  that  had  been  hired  into  these 
occupations.  With  only  one  ATCS  (FSS)  and  one  CS  ASC  HIRE  member,  no 
meaningful  information  could  be  provided.  Although  144  TRAD  HIRES  had  agreed 
to  participate  in  the  project  evaluation  from  the  ATCS  (FSS)  occupation, 
there  were  no  TRAD  HIRE  CSs.  Briefly,  the  ATCS  (FSS)  qualified  on  the  basis 
of  a  non-Aviation  major  and  relevant  work  experience  during  or  following  his 
education.  The  CS  ASC  HIRE  was  an  Aviation  Management  graduate. 

Data  Sources.  Table  3  describes  the  questionnaires  and  supporting  databases 
used  in  this  project.  With  the  exception  of  the  Biographical  Questionnaire, 
all  other  instruments  were  developed  specifically  to  support  the  evaluation 
of  the  Airway  Science  Curriculum  Demonstration  Project.  Readers  are  referred 
to  Appendix  D  of  the  complete  summative  evaluation  report  (Clough,  1987)  for 
more  detailed  information  on  the  development  and  psychometric  characteristics 
of  the  questionnaire  subscales. 

Procedure 


Participation  in  this  project  was  obtained  by  the  signing  of  consent  forms  by 
willing  new  hires  in  the  occupations  of  ATCS,  ET,  ASI,  and  CS.  All  ATCS  new 
hires  received  the  participation  consent  form  during  ATCS  Indoctrination; 
Civil  Aeromedical  Institute  (CAMI)  staff  described  the  project  and  requested 
participation.  Regional  human  resources  management  offices  were  responsible 
for  obtaining  the  consent  of  all  ASC  HIRES  and  for  requesting  the 
participation  of  non-ATCS  TRAD  HIRES.  Once  a  participation  consent  form  was 
received  by  the  Human  Resources  Research  Branch  of  CAMI,  the  individual  was 
entered  into  the  Airway  Science  databases,  career  tracking  began,  and 
questionnaires  were  distributed  as  needed. 

The  current  version  of  the  Biographical  Questionnaire  (BQ)  as  well  as  the 
Supplement  to  the  BQ  have  been  distributed  since  July  1985.  Mailings  of  the 
other  questionnaires  were  initiated  in  November  1986.  At  that  time,  an 
attempt  was  made  to  "catch  up"  and  therefore  all  questionnaires  that  should 
have  been  mailed  to  the  participant  and  his  or  her  supervisor  on  or  prior  to 
November  were  sent.  The  preferred  mailing  date  is  triggered  by  the 
entry-on-duty  date  which  corresponds  to  the  individual's  Airway  Science 
project  participation.  Follow-up  mailings  to  non-respondents  and  mailings 
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being  initiated  following  November  are  being  sent  on  this  "Airway  Science 
date."  Questionnaires  are  mailed  to  appropriate  respondents  and  supervisors 
on  a  monthly  basis . 

RESULTS 


Only  statistically  significant  differences  were  documented  in  this  paper. 
When  a  sample  size  was  1000  or  more,  a  "p  value"  of  at  least  .01  was  required 
to  describe  a  finding  as  statistically  significant.  With  smaller  sample 
sizes,  the  cutoff  "p  value"  was  .05.  The  statistical  power  was  sufficient 
with  relatively  large  samples  to  warrant  a  more  stringent  requirement.  With 
a  "p  value"  of  .05  and  over  1000  members  in  the  sample,  many  findings  were 
statistically  significant  that  were  of  questionnable  practical  significance. 

Project  Objective  U_  To  Assess  the  Unique  Impact  of  an 
FAA  Recognized  Ai rway  Science  Degree 

This  objective  focuses  exclusively  on  the  perceptions  and  behaviors  of  Airway 
Science  graduates  as  compared  with  those  who  do  not  have  an  Airway  Science 
degree.  Unfortunately,  in  May  1987,  no  reliable  mechanism  was  in  place  to 
identify  Airway  Science  graduates  who  entered  the  agency  as  a  result  of  the 
traditional  (rather  than  Airway  Science)  selection  approach.  When  sufficient 
data  are  available,  two  specific  areas  will  be  investigated  in  support  of 
this  objective.  (At  this  time,  the  number  of  Airway  Science  graduates  is  too 
few  to  compute  meaningful  statistics.)  First,  it  was  hypothesized  that  Airway 
Science  graduates  might  be  more  committed  to  a  career  in  aviation.  Second, 
and  related,  it  was  hypothesized  that  less  attrition  might  be  evident  with 
Airway  Science  graduates  than  with  others. 

Two  types  of  attrition  are  possible  and  both  will  be  examined.  Specifically, 
an  individual  may  change  occupations  but  remain  in  the  FAA;  this  is  referred 
to  as  occupational  attrition.  The  second  type  is  agency  attrition  which  is 
exit  from  the  FAA.  One  hypothesis  is  that  all  attrition  may  be  lower  with 
Airway  Science  graduates.  On  the  other  hand,  if  occupational  attrition  is 
necessary  (e.g.,  due  to  a  failure  in  training),  it  may  not  result  in  agency 
attrition  because  the  individuals  may  be  more  committed  to  remaining  with  the 
FAA  and  may,  due  to  their  varied  backgrounds,  be  qualified  for  multiple 
positions. 

In  addition  to  the  two  areas  mentioned  above,  this  objective  also  provides 
the  basis  for  a  focused  analysis  of  Airway  Science  graduates  within  the 
larger  experimental  groups  of  ASC  HIRE5  for  project  Objectives  3  through  8. 
It  is  important  to  recognize  the  significance  of  this  objective  to  the 
long-term  project  evaluation.  It  is  expected  that  the  number  of  Airway 
Science  graduates  selected  into  the  FAA  occupations  of  interest  will  be 
sufficiently  large  to  permit  an  examination  of  this  objective  in  the  future. 

Project  Objective  2 :  To  Enhance  FAA  Ties  wi th 

Universities/Coll eges  Offering  Aviation-Related  Degrees 

The  data  relevant  to  this  discussion  were  compiled  in  April  1987  by  the  UAA 
from  its  Airway  Science  Curriculum  database.  At  that  time,  there  were  29 
colleges  and  universities  across  the  country  with  recognized  Airway  Science 


10 


programs.  One  additional  school  previously  had  an  FAA  recognized  program. 
That  recognition  was  suspended,  however,  when  it  was  discovered  that  the 
program  had  not  been  implemented  as  proposed;  the  school  was  in  the  process 
of  working  to  reestablish  its  prior  status.  The  majority  of  Airway  Science 
programs  received  FAA  recognition  when  the  Demonstration  Project  began  in 
1983  (see  Figure  1).  Twenty-four  of  the  30  institutions  had  Airway  Science 
Management  degree  programs.  In  addition,  20  and  18  schools,  respectively, 
had  recognized  Airway  Computer  Science  and  Aircraft  Systems  Management 
programs.  There  were  26  colleges  and  universities  with  recognition  in  more 
than  one  area  of  concentration  suggesting  strong  support  for  the  Airway 
Science  program  by  participating  schools.  In  fact,  10  institutions  had 
returned  to  the  FAA  to  request  recognition  in  areas  of  concentration  in 
addition  to  those  originally  proposed. 

Perhaps  a  more  comprehensi ve  index  of  university  support  for  the  Airway 
Science  program  is  attained  by  examining  the  number  of  Airway  Science 
curriculum  proposal  submissions.  As  previously  mentioned,  institutions 
interested  in  receiving  FAA  recognition  of  their  Airway  Science  program  must 
successfully  complete  an  extensive  proposal  review.  The  curriculum  proposal 
review  process  was  designed  to  ensure  the  academic  integrity  of  the  Airway 
Science  program.  Submission  of  a  curriculum  proposal  represents,  from  the 
FAA 1 s  perspective,  a  valid  measure  of  interest  in  the  Airway  Science  program 
since  even  curriculum  proposal  development  requires  an  institutional 
commitment  of  time  and  other  resources. 

Figure  2  illustrates  the  number  of  Airway  Science  proposals  received  in  each 
area  of  concentration  (AOC)  from  1982  through  1987.  In  these  statistics,  one 
school  applying  for  five  AOCs  counts  five  times  (one  per  AOC).  However,  a 
given  curriculum  proposal  (regardless  of  the  number  of  times  it  may  be 
resubmitted)  counts  only  once.  The  number  of  proposal  submissions  has  varied 
substantially  from  one  year  to  the  next  with  a  high  of  51  in  1983  and  a  low 
of  eight  in  1986.  During  1987  (as  of  April),  19  proposals  had  been  received 
from  nine  schools.  Considering  that  only  4  months  had  elapsed  in  1987,  the 
proposal  submission  rates  were  up  markedly  (see  also  Figure  3).  The  number 
of  1987  proposal  submissions  for  four  of  the  five  AOCs  was  already  higher 
than  that  of  the  previous  year;  the  number  of  1986  and  1987  submissions  was 
identical  for  the  remaining  AOC  (see  Figure  2).  [Note:  Figure  3  differs 
from  the  preceding  two  charts  in  that  institutions  rather  than  AOCs  were 
counted,  and  only  initial  proposal  submission  and  recognition  were  recorded.] 

Several  hypotheses  have  been  suggested  to  explain  the  proposal  submission  and 
recognition  levels  previously  described.  The  majority  of  institutions 
receiving  Airway  Science  program  recognition  in  the  first  2  years  had 
longstanding  aviation  programs  which  required  relatively  minor  modifications 
to  accommodate  the  Airway  Science  degree  requirements  (G.  W.  Kiteley, 
personal  communication.  May  1987).  More  recently,  more  of  the  schools 
submitting  curriculum  proposals  (some  of  which  eventually  have  achieved  FAA 
recognition)  have  developed  their  Airway  Science  programs  without  a  strong 
background  in  aviation.  Their  involvement  in  Airway  Science  represents  an 
attempt  to  initiate  aviation  education  within  their  institution. 

An  FAA  program  which  probably  contributed  to  the  fluctuations  in  proposal 
submission  rates  was  the  Airway  Science  Grant  Program  which  was  designed  to 
provide  selected  institutions  with  partial  funding  for  the  enhancement  of 
their  Airway  Science  program.  Specifically,  awarded  funds  can  be  used  to 
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Figure  3.  Institutional  Activities  Per  Year:  First  Proposal  Submission 
Versus  Initial  Recognition 
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support  the  purchase,  lease,  or  construction  of  buildings  or  associated 
facilities,  or  non-expendable  instructional  materials  or  equipment  and  are 
to  be  used  in  conjunction  with  an  FAA  recognized  Airway  Science  curriculum. 
During  CY-84,  the  grant  proposal  sol icitation  was  limited  to  Historically 
Black  Colleges  and  Universities  (HBCUs);  interested  HBCUs  were  required  to 
submit  an  Airway  Science  curriculum  proposal  to  the  FAA  prior  to  submitting  a 
grant  proposal.  During  CY-85,  all  institutions  that  had  submitted  a 
curriculum  proposal  prior  to  October  16,  1985,  were  eligible  to  compete  for 
grant  monies.  The  relatively  high  number  of  curriculum  proposals  submitted 
in  1985  may  have  been  due,  in  part,  to  the  grant  solicitation.  The  CY-87 
competitive  grant  award  was  restricted  to  those  institutions  with  recognized 
Airway  Science  programs  prior  to  December  31,  1986. 

It  is  unlikely  that  the  majority  of  institutions  have  become  involved  in  the 
Airway  Science  program  solely  because  of  the  availability  of  federal  funds. 
As  previously  discussed,  the  curriculum  review  process  alone  is  very  thorough 
and  difficult.  In  addition,  institutions  must  totally  support  human  and 
other  resources  to  initiate,  develop,  and  maintain  their  Aww ay  Science 
program.  The  Grant  Program  may  have  served  as  a  catalyst  to 
colleges/universities  with  an  interest  in  aviation,  encouraging  them  to 
institute  their  own  Airway  Science  degree  program.  The  Demonstration  Project 
and  the  Airway  Science  Grant  Program  both  serve  to  complement  the  FAA1 s 
objective  in  enhancing  its  ties  with  aviation  higher  education. 

Project  Objective  3^_  To  Maintain  the  Technical  Competence  of  the 

FAA  Workforce  During  a  Period  of  Rapid  Technological  Change 

Occupations  were  examined  separately.  At  this  point,  technical  competence 
was  assessed  via  training  activities  and  performance.  As  might  be  expected, 
the  training  and  the  manner  in  which  performance  was  measured  differ 
substantially  between  occupations.  It  should  be  recognized  that  technical 
training  within  the  FAA  is  not  all  classroom  instruction,  but  rather  involves 
extensive  work  (both  on-the-job  training  and  independent)  in  the  field. 
Although  the  Human  Resources  Research  Branch  maintains  data  on  ATCS  (T/E) 
field  training,  this  database  has  not  yet  been  fully  integrated  with  the 
Airway  Science  project  evaluation  database.  Therefore,  the  focus  of  the 
present  discussion  is  on  performance  in  the  non-radar  screening  program  which 
is  administered  to  new  trainees  at  the  FAA  Academy  upon  entry  on  duty.  ATCS 
students  must  pass  this  screen  to  remain  in  ATCS  (T/E)  training.  For  the 
purposes  of  the  Airway  Science  project  evaluation,  pass  rates  are  determined 
on  the  basis  of  the  trainee's  last  entrance  into  the  FAA  Academy.  For 
example,  the  second  pass/  fail /withdrawal  outcome  would  be  the  one  used  to 
compute  the  pass  rate  for  trainees  who  recycle  through  the  program.  This 
strategy  for  determining  pass  rates  was  selected  because  most  of  the  interest 
in  the  Airway  Science  program  evaluation  is  focused  on  tracking  the  success 
of  individuals. 


ATCS  (T/E)  Training  Performance 

The  initial  qualification  training  of  ATCSs  (T/E)  was  altered  substantially 
following  the  class  which  ended  in  December  1985.  Prior  to  1986,  incumbents 
were  placed  in  the  terminal  or  en  route  options  before  beginning  the  program 
with  a  unique  initial  pass/fail  screen  for  each  option.  With  the  class  that 
began  training  in  October  198^  (and  graduated  in  January  1986),  a  common 


initial  qualification  screen  was  established  and  terminal/en  route  option 
placement  was  moved  to  after  the  pass/fail  screen.  As  a  result  of  the 
changes  to  the  pass/fail  screen  program,  the  former  terminal  and  en  route 
screens  and  the  current  "common  screen"  must  be  examined  separately.  One 
component  of  the  screen  which  remained  consistent  during  the  transition  was 
Phase  II.  This  was  exclusively  classroom  instruction  designed  to  provide 
students  with  basic  knowledge  of  aviation  and  air  traffic  control.  In 
comparing  the  performance  of  ASC  and  TRAD  HIRES  in  Phase  II,  no  differences 
on  the  basis  of  hiring  status  were  observed. 

Former  Terminal  and  En  route  Screen  Programs.  Since  there  appeared  to  be  no 
pattern  in  the  assignment  of  ASC  and  TRAD  HIRES  into  terminal  and  en  route 
screens,  the  two  screens  were  combined  for  the  purposes  of  analysis.  Thirty- 
five  ASC  HIRES  and  323  TRAD  HIRES  participated  in  the  project  evaluation.  As 
Table  4  indicates,  the  pass  rate  of  ASC  HIRES  was  significantly  lower  than 
that  of  TRAD  HIRES  (x2  =  10.53,  p  <  .01). 


Table  4.  Pass/Fail  Rates  in  Non-Radar  Training 
for  Airway  Science  and  Traditional  Hires 
(as  of  May  12,  1987) 


FORMER  TERMINAL  AND  EN  1 

ROUTE 

Pass 

Fail 

Withdraw/ 

Total 

PROGRAMS: 

Incomplete 

n 

% 

n 

% 

n 

% 

n 

% 

Airway  Science  Hires 

15 

42.9 

19 

54.3 

1 

2.9 

35 

100.0 

Traditional  Hires 

211 

65.3 

90 

27.9 

22 

6.8 

323 

100.0 

CURRENT  COMMON  SCREEN  PROGRAM: 

Airway  Science  Hires 

27 

44.3 

27 

44.3 

7 

11.5 

61 

100.0 

Traditional  Hires 

1358 

65.2 

596 

28.6 

128 

6.1 

2082 

100.0 

Common  Screen  Program.  Sixty-one  ASC  HIRES  and  2082  TRAD  HIRES  had  completed 
the  screen  as  of  May  12,  1987.  As  Table  4  suggests,  the  pass  rate  of  ASC 
HIRES  was  significantly  lower  than  that  of  TRAD  HIRES  (x2  =  11.68,  p  <  .005). 
It  would  be  useful  to  look  at  the  relative  performances  of  the  various 
subgroups  of  ASC  HIRES  as  classified  by  the  eligibility  scheme  previously 
described;  however,  cell  sizes  were  too  small  to  identify  any  trends.  Since 
it  is  important  to  determine  whether  specific  subgroups  of  ASC  HIRES  are 
performing  much  worse  (or  better)  than  the  others,  larger  sample  sizes  are 
necessary  to  fully  assess  ATCS  training  performance. 

Components  of  the  non-radar  screen  were  examined  to  identify  those  aspects  of 
the  program  responsible  for  the  lower  ASC  HIRES  performance.  Briefly,  It  was 
observed  that  ASC  HIRES  scored  significantly  lower  on  the  laboratory 
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simulation  problems  as  measured  by  both  the  average  technical  and  instructor 
assessments  (F  (1,1983)  =  13.92  p  <  .0005;  F  (1,1983)  =  15.60,  p  <  .0001). 
In  addition,  the  average  score  of  ASC  HIRES  on  the  non-radar  Laboratory 
Controller  Skills  Test  was  almost  nine  points  lower  than  that  of  TRAD  HIRES 
(F  (1,1981)  =  26.33,  p  <  .0001). 

Two  predictors  of  non-radar  screen  performance  which  have  been  established 
from  years  of  research  at  the  Civil  Aeromedical  Institute  are  age  at  the  time 
of  training  and  performance  on  the  Office  of  Personnel  Management  Air  Traffic 
Controller  Selection  Test  (Collins,  Boone,  VanDeventer,  1981;  P.  Kegg, 
personal  communication,  June  1987).  Generally,  younger  people  are  more 
likely  to  succeed  in  training,  as  are  individuals  who  score  higher  on  the 
Selection  Test.  [It  was  found  that  the  average  ages  of  ASC  and  TRAD  HIRES  at 
the  time  that  they  entered  the  non-radar  screen  did  not  differ.] 

The  typical  selection  approach  is  based  solely  on  performance  on  the  ATC 
Selection  Test.  In  a  sense,  the  Airway  Science  hiring  program  is  being 
compared  to  the  Selection  Test  with  regard  to  its  predictive  power.  Since 
only  107.  of  the  ASC  HIRE  rating  is  based  on  the  Selection  Test,  it  is  not 
surprising  that  ASC  HIRES  scored  significantly  lower  than  TRAD  HIRES  on  the 
test  (F  (1,  2021)  =  61.09,  p  <  .0001).  In  an  attempt  to  determine  whether 
the  lower  test  performance  could  be  a  factor  in  the  lower  non-radar  screen 
performance,  an  hierarchical  regression  was  completed  in  which  any  variation 
in  the  screen  final  score  which  was  explained  by  performance  on  the  ATC 
Selection  Test  was  statistically  removed.  The  differences  between  ASC  and 
TRAD  HIRE  non-radar  screen  performance  disappeared  (see  Table  5).  This 
finding  should  not  be  interpreted  as  a  recommendation  to  abolish  the  Airway 
Science  ATCS  selection  standards.  However,  this  aspect  of  the  hiring  program 
merits  close  examination  as  the  number  of  ASC  HIRES,  particularly  the  grossly 
underrepresented  Airway  Science  graduates,  increases. 

Performance  in  the  non-radar  screen  is  just  one  aspect  of  an  ATCS's  training. 
It  was,  and  will  continue  to  be,  closely  monitored  since  it  operates  as  the 
initial  screening  mechanism  for  entrance  into  the  field  of  air  traffic 
control  in  the  terminal  and  en  route  environments.  However,  field  training 
also  is  an  important  measure  of  the  technical  competence  of  ATCS  incumbents. 
The  capacity  of  ASC  HIRES  as  compared  with  TRAD  HIRES  to  perform  in  the  field 
will  be  investigated  in  the  future. 


Table  5.  Hierarchical  Regression  of  Selection  Criteria  on 
Official  Non-Radar  Lab  Composite  Score 


Step 

Selection  Measure 

R2 

6 

t 

1. 

Transmuted  Composite  ATC  Selection  Test 

.059 

.243 

10.68  **** 

2. 

Hiring  Status  (Airway  Science  versus 
Traditional  Hire) 

.059 

.011 

0.51 

**** 

p  <  .001 
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ET  Training  Performance 


To  obtain  information  on  the  training  activities  of  project  evaluation 
participants,  a  Training  Progress  Report  was  sent  to  supervisors  of 
participants  with  one  or  more  years  in  the  agency.  Responses  were  received 
for  12  ASC  HIRES  and  12  TRAD  HIRES.  These  sample  sizes  were  too  small  to 
analyze  categorical  data  statistically;  therefore,  the  findings  are  reported 
for  descriptive  purposes  only.  The  majority  of  ETs,  regardless  of  hiring 
status,  completed  at  least  one  directed  study/correspondenre  course  within 
their  first  year;  half  of  the  ASC  HIRES  and  slightly  more  (63.6%)  TRAD  HIRES 
received  Computer-Based  Instruction;  40%  of  TRAD  HIRES  and  43%  of  ASC  HIRES 
completed  one  or  more  field  courses;  and  58.3%  of  ASC  HIRES  and  63.6%  of  TRAD 
HIRES  enrolled  in  one  or  more  resident/  Academy  courses.  Only  one  TRAD  HIRE 
failed  any  coursework.  Twenty-five  percent  of  ASC  HIRES  successfully 
completed  one  or  more  performance  examinations  whereas  two-thirds  of  TRAD 
HIRES  completed  at  least  one. 

The  training  progress  of  most  participants  was  thought  to  be  on  schedule  or 
ahead  of  schedule  (see  Table  6).  Again,  this  information  is  provided  as 
background  on  how  the  two  samples  are  performing  and  does  not  imply  how  ASC 
and  TRAD  HIRES  will  do  in  the  future.  Finally,  supervisors  were  asked  to: 
(a)  characterize  the  participant's  ability  to  apply  coursework;  (b)  assess 
the  pace  of  the  person's  OJT  progress;  (c)  evaluate  his  or  her  ability  to 
aoply  what  was  learned  in  OJT;  and  (d)  rate  the  person's  training  performance 
as  compared  with  all  other  ETs.  Although  the  sample  sizes  are  small, 
statistical  comparisons  of  the  rating  data  using  ANOVA  were  possible.  The 
ratings  were  completed  using  a  five-point  scale  in  which  a  rating  of  "3''  is 
average.  In  all  cases,  both  ASC  and  TRAD  HIRES  average  ratings  are  at  least 
3.0  suggesting  that  neither  group  is  subpar.  It  was  found  that  the  pace  of 
the  ASC  HIRE  sample's  OJT  progress  (M  =  4.17)  was  rated  as  slightly  (but 
significantly)  faster  than  that  of  TRAD  HIRES  (M  =  3.42)  (F  (1,22)  =  5.09,  p 
<  .05). 

ASI-OP  Training  Performance 

Training  Progress  Reports  were  also  administered  for  both  ASI  occupations. 
Data  were  available  for  12  ASC  HIRES  and  27  TRAD  HIRES.  Within  the  past 
year,  the  majority  of  participants  completed  at  least  one  resident/Academy 
course  (ASC:  100.0%;  TRAD:  65.4%);  received  OJT  (ASC:  100.0%;  TRAD: 
66.7%);  and  worked  independently  (no  longer  receiving  OJT)  on  at  least  one 
major  job  function/task  (ASC:  66.7%;  TRAD:  92.6%).  Again,  this  information 
cannot  be  analyzed  statistically  at  this  point  but  is  provided  to  describe 
the  training  activities  of  participants.  Most  participants  were  either 
consistent  with  or  ahead  of  what  was  expected  with  regard  to  their  training 
progress  (see  Table  6).  ASI  supervisors  were  asked  to  rate  the  same  aspects 
of  training  performance  as  ET  supervisors.  The  only  statistically 
significant  difference  between  ASC  and  TRAD  HIRES  was  with  the  perceived  pace 
of  OJT  progress  (F  (1,35)  =  5.02,  p  C.05).  ASC  HIRES  were  perceived  as 
moving  at  a  somewhat  slower  pace  (M  =  3.67)  than  TRAD  HIRES  (M  =  4.32) 
although  the  ASC  HIRES  average  rating  on  this  item  was  acceptable. 

AS  I -AW  Training  Performance 

Training  Progress  Report  responses  were  received  for  7  ASC  HIRES  and  50  TRAD 
HIRES.  Once  again,  information  is  provided  only  for  descriptive  purposes  and 
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Table  6.  Training  Progress  of  Airway  Science  and  Traditional  Hires 
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Note.  ET  -  tiectronics  Technician;  ASI-OP  -  Aviation  Safety  Inspector  -  Operations;  ASI-AW  -  Aviation  Safety 
Inspector  -  Airworthiness;  AWS  -  Airway  Science  Hire;  TRAD  -  Traditional  Hire 


not  to  assess  group  differences  between  ASC  and  TRAD  HIRES.  As  with  the 
ASI-OP  occupation,  most  participants  had  completed  one  or  more 
resident/Academy  courses  (ASC:  6  of  7  (85.7%);  TRAD:  84.0%);  received  OJT 
(ASC:  „  of  6  (83.3%);  TRAD:  90.0%)  and  were  able  to  work  independently 
(ASC:  4  of  6  (66.6%);  TRAD:  88.0%).  The  majority  of  ASC  HIRES  were 
described  as  ahead  of  what  was  expected  with  regard  to  their  training 
progress  (see  Table  6).  ASC  HIRES  were  rated  significantly  lower  than  TRAD 
HIRES  in  their  ability  to  apply  what  had  been  learned  in  OJT;  however,  once 
again,  even  the  average  rating  of  ASC  HIRES  was  acceptable  (M  =  3.4). 

Project  Objective  4j_  To  Develop  Within  the  FAA  Workforce  an 

Increased  Acceptance  of  Technological  Change 

The  data  to  respond  to  this  objective  came  from  two  sources:  the 
Biographical  Questionnaire  in  which  the  incumbents'  expectations  regarding 
increased  automation  were  asked  and  the  Status  Survey  which  was  sent  to 
employees  after  being  with  t*'e  agency  for  one  year  or  more.  Due  to  missing 
data,  the  expectation  questions  were  analyzed  for  only  ATCS  (T/E).  No 
statistically  significant  differences  were  observed  between  ASC  HIRES  and 
TRAD  HIRES  with  regard  to  the  likelihood  that:  (a)  greater  automation  will 
occur  in  the  future;  or  (b)  negative  feelings  will  result  from  increased 
automation  in  the  workplace. 

The  remaining  questions  were  asked  of  those  who  had  been  in  their  positions 
for  one  or  more  years;  all  four  occupational  groups  were  considered. 
Regarding  the  respondents’  predictions  that  increased  automation  will  affect 
job  tasks  in  the  future,  when  examining  differences  between  ASC  and  TRAD 
HIRES  within  each  occupation,  ASI-OP  ASC  HIRES  predicted  less  likelihood  of 
change  than  ASI-OP  TRAD  HIRES  (F  (3,182)  -  3.01,  p  <  .05).  All  other 
intra-occupational  differences  were  non-significant.  The  only  observed  group 
differences  occurred  with  the  four  occupational  groups  and  their  feeling  that 
increased  automation  would  influence  their  effectiveness.  Of  interest  in  the 
project  evaluation  are  differences  between  ASC  and  TRAD  HIRES  in  general,  and 
then  within  each  occupation  if  the  main  and  interaction  effects, 
respectively,  are  statistically  significant;  general  differences  between  the 
occupations  as  a  whole  are  not  relevant  to  this  re.iew.  No  differences  were 
found  when  considering  respondents'  feeling  that  increased  automation  would 
affect  job  satisfaction  or  job  challenge.  In  summary,  hiring  status  had  very 
little  influence  on  expectations,  knowledge,  or  feelings  regarding  increased 
automation  at  work. 

Project  Objective  5j_  To  Recruit  and  Hire  Individuals 
wi th  Managerial  Potential 

The  Managerial  Skills  Scale  (MSS)  was  designed  to  identify  the  extent  to 
which  project  participants  demonstrate  the  skills  linked  to  effective  human 
resource  management.  Also,  respondents  (the  participants'  supervisors)  were 
asked  to  predict  whether  the  participant  has  the  potential  at  some  point  to 
be  a  supervisor  and  when  that  might  be.  (See  Appendix  D  in  Clough,  1987  for 
a  thorough  description  of  the  contents  and  psychometric  characteristics  of 
th?  scales  described  in  this  and  later  sections.) 

Conclusions  drawn  from  the  MSS  survey  must  be  considered  preliminary  since: 
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(a)  most  hires  after  their  first  year  are  still  in  training  status  (ATCS 
supervisors  in  particular  responded  negatively  to  evaluating  the  management 
potential  of  an  individual  who  has  not  yet  been  able  to  demonstrate  whether 
he  or  she  is  capable  of  being  a  competent  controller);  and  (b)  in  the  ASI 
profession  especially,  ASC  HIRES  enter  the  agency  at  a  much  lower  grade  than 
the  majority  of  TRAD  HIRES  and  therefore  might  be  expected  to  have  less 
opportunity  to  demonstrate  supervisory  skills  than  their  TRAD  HIRES 
counterparts . 

As  expected,  there  were  significant  differences  between  the  managerial  skills 
of  the  ASC  HIRES  and  TRAD  HIRES  samples  within  certain  occupations  (as 
measured  by  the  MSS)  (F  (3,675)  =  2,90,  p  <  .05).  ET  ASC  HIRES  were 
perceived  as  showing  greater  skills  than  ET  TRAD  HIRES.  This  measure  is 
likely  to  become  increasingly  accurate  as  each  supervisor  has  a  longer  time 
to  interact  with  his  or  her  employees. 

The  next  set  of  items  was  contained  in  the  Status  Survey  and  was  answered  by 
participants  with  one  or  more  years  in  the  FAA.  No  significant  differences 
were  found  with  regard  to  the  extent  that  respondents:  (a)  felt 
i ntel 1 ectual ly  challenged  by  their  jobs;  (b)  perceived  advancement 
opportunities;  (c)  indicated  that  becoming  a  manager  or  supervisor  was 
personally  important;  or  (d)  were  satisfied  with  their  chances  of  receiving  a 
promotion . 

Project  Object i ve  6j_  To  Attract  Individuals  wi th  Greater  Awareness 

of  and  Skills  i_n  Maintaining  Positive  Human  Relations 

Relevant  Biographical  Questionnaire  items  referred  to  the  importance  of  and 
expectations  regarding  skill  utilization,  participative  decision-making,  and 
other  intrinsic  factors  of  the  job.  Selected  expectation  questions  were 
combined  to  create  the  Work-Related  Challenge  Scale  to  address  aspects  of 
this  issue.  Again,  the  data  were  analyzed  using  only  the  ATCS  ( T/E )  sample 
and  no  significant  differences  on  the  basis  of  hiring  status  were  found.  In 
fact,  both  ASC  and  TRAD  ATCS  expected  to  be  challenged  by  the  job;  the 
average  response  for  the  two  groups  was  4.1  on  a  five-point  scale. 
Similarly,  the  occupational  choice  items  of  intellectual  challenge,  autonomy, 
opportunity  to  work  with  competent  people,  ability  to  control  workload,  and 
prestige  of  the  job  were  examined  and  were  found  to  be  unrelated  to  hiring 
status.  Because  of  the  agency's  strong  concern  with  the  human  relations 
skills  of  its  supervisors,  several  ot  the  questions  previously  discussed 
under  Objective  5  are  relevant.  The  MSS  includes  interaction  skills  such  as 
oral  communication  and  interpersonal  skills.  No  significant  differences  were 
observed  with  these  individual  ratings. 

Several  questions  on  the  Status  Survey  also  pertain  to  an  interest  in  or 
feelings  toward  human  relations  concepts.  It  was  found  that  ASC  HIRES  as  a 
group  perceived  slightly  less  autonomy  in  their  jobs  (M  =  2.88)  than  did  TRAD 
HIRES  (M  =  3.42)  (F  (1,186)  =  6.40,  p  <  .05).  Although  the  Hiring  Status  x 
Occupation  interaction  overall  was  not  statistically  significant,  the  above 
finding  was  especially  apparent  with  ASI-OP  participants  wherein  ASC  HIRES 
had  an  average  rating  more  than  one  point  lower  than  the  average  of  TRAD 
HIRES.  In  reflecting  on  the  entry  grades  (and  probable  experience)  of  the 
two  groups,  differences  in  actual  work  assignments  may  account  for  the 
previous  finding.  Also,  ATCS  ASC  HIRES  assigned  slightly  less  importance  to 
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having  tasks  that  used  many  skills  and  abilities  (M  =  3.22)  than  did  their 
TRAD  HIRES  counterparts  (M  =  3.92)  (F  (3,186)  =  2.65,  p  =  .05);  the  remaining 
differences  within  each  occupation  were  non-significant.  No  differences 
between  ASC  and  TRAD  HIRES  were  observed  for  the  following  perceptions 
regarding  the  extent  to  which  each  was  provided  by  their  FAA  job:  (a) 
opportunity  to  work  with  competent  people;  (b)  tasks  that  require  a  number  of 
skills  and  abilities;  or  (c)  the  opportunity  to  make  a  substantial  impact  on 
the  lives  of  others.  In  addition,  the  following  factors  were  rated  as  no 
more  important  to  one  group  than  the  other:  (a)  intellectual  challenge;  (b) 
autonomy;  (c)  working  with  competent  people;  (d)  the  opportunity  to  make  an 
impact  on  the  lives  or  work  of  others;  or  (e)  the  opportunity  to  contribute 
to  decisions  that  affect  one's  job. 

Project  Objective  7j_  To  Increase  Female  and  Minority 
Representation  Within  the  FAA 

There  was  insufficient  data  to  adequately  address  this  issue.  Statistics 
comparing  the  demographics  of  ASC  and  TRAD  HIRES  were  possible  only  with  the 
ATCS  (T/E)  sample.  No  differences  in  the  representation  of  women  and 
minorities  were  observed.  However,  in  both  ASI-OP  and  ASI-AW,  the 
percentages  of  women  and  minorities  in  the  ASC  HIRES  sample  were 
substantially  higher  than  those  of  TRAD  HIRES  (refer  to  the  relevant  Project 
Participant  sections  earlier  in  this  report).  Again,  without  statistical 
comparisons,  it  cannot  be  determined  whether  this  was  likely  to  be  a 
consistent  pattern  or  merely  due  to  chance.  Clearly  this  is  an  aspect  of  the 
project  that  requires  close  examination  in  the  future. 

Project  Objective  8^  To  Improve  Employee  Perceptions  of  the  FAA 

This  project  objective  is  designed  to  examine  feelings  that  employees  have 
regarding  their  job,  supervisor,  and  coworkers.  Several  questions  from  the 
Biographical  Questionnaire  relate  to  expectations  regarding  satisfaction  with 
general  work  factors.  Four  satisfaction  expectation  scales  were  developed, 
two  of  which  were  relevant  to  this  objective.  Again,  these  data  were 
examined  only  for  ATCS  (T/E).  No  differences  were  found  with  regard  to  the 
extent  to  which  participants  expected  to  be  satisfied  with  the  "basics" 
(e.g.,  salary,  benefits)  as  measured  by  the  Basics  Scale.  In  addition,  the 
Scale  of  Management  Focus,  which  measured  the  extent  to  which  employees 
believe  in  the  FAA  management  structure,  was  developed.  No  statistically 
significant  differences  were  observed  between  ASC  and  TRAD  HIRES. 

Of  greater  interest  is  the  reported  satisfaction  of  employees  with  experience 
in  the  agency.  Five  factors  were  identified  from  the  satisfaction  items  in 
the  Status  Survey.  The  resulting  scales  tapped  general  job  satisfaction, 
supervisor  satisfaction,  coworker  satisfaction,  and  personal  control.  No 
statistically  significant  differences  between  ASC  and  TRAD  HIRES  were 
observed  with  the  General  Job  Satisfaction  Scale,  Supervisor  Satisfaction 
Scale,  or  the  Coworker  Satisfaction  Scale.  With  regard  to  pay  satisfaction, 
ASI-OP  ASC  HIRES  reported  slightly  less  satisfaction  with  their  pay  (M  = 
2.00)  than  did  their  TRAD  HIRE  counterparts  (M  =  2.94)  (F  (3,186)  =  3.58,  p  < 
.05).  No  differences  due  to  hiring  status  were  significant  with  the  other 
occupations. 
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DISCUSSION 


As  stated  at  the  outset,  the  purpose  of  this  report  was  to  describe  the 
status  of  the  Airway  Science  program  relative  to  its  eight  objectives  shortly 
before  the  end  of  its  fourth  year  of  implementation.  This  report  also  served 
as  the  basis  for  the  agency's  request  to  the  Office  of  Personnel  Management 
for  an  extension  to  the  5  year  demonstration  project.  Issues  addressed 
include  global  intentions  of  the  program  such  as  the  desire  to  enhance  the 
ties  with  collegiate  aviation  education  as  well  as  areas  of  concern  specific 
to  the  FAA  such  as  recruiting  and  selecting  individuals  with  managerial 
potential  and  a  readiness  to  accept  technological  change.  At  this  stage  of 
the  program,  few  consistent  differences  were  observed  between  Airway  Science 
and  traditional  hires. 

Within  this  evaluation  report,  two  aspects  of  the  Airway  Science  project  were 
noteworthy.  First,  interest  on  the  part  of  the  academic  community  was 
evident  and  appeared  to  be  strong.  Although  the  majority  of  institutions 
with  recognized  Airway  Science  programs  received  FAA  recognition  in  the  first 
year  of  the  project,  submission  rates  for  curriculum  proposals  appeared  to  be 
up  markedly  in  1987.  In  addition,  it  has  been  suggested  that  the  type  of 
academic  institution  pursuing  participation  in  the  Airway  Science  program  may 
be  changing  to  some  extent.  Within  the  past  several  years,  proportionally 
more  of  the  colleges  and  universities  submitting  Airway  Science  curriculum 
proposals  were  relatively  new  to  aviation  education.  Thus,  the  Airway 
Science  program  may  be  promoting  the  growth  of  collegiate  aviation  education. 

The  second  finding  that  should  be  mentioned  is  the  relatively  lower  pass 
rates  of  Airway  Science  hires  as  compared  with  traditional  hires  In  the  FAA 
Academy's  air  traffic  control  (terminal/en  route)  screen  program. 
Preliminary  evidence  suggests  that  the  lower  pass  rate  may  be  linked  to  lower 
scores  on  the  0?M  ATC  Selection  Test.  Since  test  performance  accounts  for 
only  10%  of  a  candidate's  Airway  Science  rating,  a  higher  weighting  of  the 
Selection  Test  may  be  warranted.  As  a  greater  number  of  Airway  Science  hires 
complete  the  program,  it  should  be  possible  to  determine  the  relative  impacts 
of  college  education  and  work  experience  that  may  relate  to  success  or 
failure  in  the  screen. 

The  original  premise  of  the  Airway  Science  program  was  that  a  broader-based 
background  may  become  increasingly  important  as  an  individual  rises  into 
supervisory  and  managerial  ranks.  The  evidence  used  to  evaluate  the  program 
to  this  point  has  largely  focused  on  technical  competence  and  new  employee 
perceptions  and  expectations  about  work.  These  short-range  measures  were  all 
the  information  that  was  available  to  date;  however,  they  do  not  adequately 
assess  the  longer  term  aspects  of  the  Airway  Science  project.  For  example, 
to  determine  project  participants'  ability  to  accept  and  adapt  to 
technological  change,  their  performance  and  perceptions  will  need  to  be 
examined  as  such  changes  occur  within  their  workplace.  Also,  as  previously 
mentioned,  it  was  thought  that  the  knowledge,  skills,  and  abilities  obtained 
through  completion  of  an  Airway  Science  degree  would  be  valuable  as 
individuals  attempted  to  bid  on  and  perform  in  supervisory  positions.  Hiring 
in  support  of  the  Airway  Science  program  began  in  1984;  it  will  be  several 
years  before  any  movement  into  supervisory  positions  can  be  anticipated. 

Finally,  it  must  be  recognized  that  only  five  Airway  Science  graduates  have 
been  selected  into  the  FAA  (others  qualified  for  the  Airway  Science  register 
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by  demonstrating  "equivalent"  college  education  or  work  experience). 
Graduates  of  the  unique  Airway  Science  curriculum  are  the  subgroup  of  Airway 
Science  hires  in  which  the  agency  is  most  interested.  Increasing  numbers  of 
Airway  Science  graduates  are  expected  to  enter  the  FAA  in  the  future. 
Academic  institutions  first  received  FAA  recognition  for  Airway  Science 
programs  in  1983.  Therefore,  students  who  were  freshmen  when  the  programs 
were  first  put  into  place  will  be  graduating  in  1987  or  1988.  Also,  approved 
refinements  to  the  Airway  Science  curriculum  review  process  should  make  the 
Airway  Science  degree  more  salient  to  undergraduate  students  interested  in  an 
aviation  career,  thereby  increasing  student  enrollments  (and  the  resulting 
appl i cant  pool ) . 

Tne  time  required  to  establish  the  Airway  Science  program,  given  its  ambition 
to  link  collegiate  aviation  education  with  entry  into  FAA  technical 
occupations  which  themselves  require  several  years  of  additional  training, 
extends  well  beyond  the  initial  5  year  demonstration  project  duration.  It  is 
recommended  that  those  with  an  interest  in  the  Airway  Science  project 
recognize  and  accept  the  limitations  of  this  preliminary  assessment.  To 
date,  graduates  of  recognized  Airway  Science  programs  have  not  been  examined 
separately  from  all  other  "look-alike"  Airway  Science  hires.  Again,  although 
very  few  Airway  Science  graduates  have  been  selected,  this  is  expected  to 
change  in  the  very  near  future  as  the  first  undergraduate  students  complete 
the  4  to  5  year  program.  In  addition,  the  project  evaluation  has  focused 
primarily  on  technical  competence  observed  in  the  first  year  of  training. 
The  Airway  Science  background  was  not  expected  to  improve  the  technical 
competence  of  the  incumbent  workforce;  rather,  a  strong  technical  orientation 
was  rejected  in  favor  of  a  relatively  broad-based  exposure  to  a  variety  of 
subject  areas  which  were  thought  to  enhance  other  aspects  of  an  employee's 
contribution  to  the  FAA  over  the  years  of  his  or  her  career.  Although 
adequate  technical  competence  is  required  to  continue  within  each  of  the 
relevant  occupations,  the  expected  pay-offs  of  the  Airway  Science  program  are 
several  years  in  the  future.  It  is  only  at  that  time  that  an  objective, 
balanced  appraisal  of  this  program  can  be  complete. 
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Appendix  A 

Airway  Science  Curriculum  Demonstration  Project  Objectives 


ORGANIZATIONAL  GOAL  #1:  To  explore  the  possibility  and  to  determine  the 
eventual  consequences  of  restructuring  the  FAA  career  system  link  with 
the  academic  community.  As  presently  envisioned,  aviation  education  and 
training  traditionally  borne  solely  by  the  FAA  will  be  shared  by  the 
individual  and  academic  institutions. 


Objective  1:  To  assess  the  unique  impact  of  a  tailor-made 

baccalaureate  curriculum  (i.e.,  an  FAA-recognized  Airway 
Science  program)  on: 

a.  One's  interest  in  pursuing  an  aviation-related  career 

b.  Occupational  and  organizational  attrition 

c.  Objectives  3  through  8 

Objective  2:  To  enhance  FAA  ties  with  universities/colleges  offering 
aviation-related  degrees 


ORGANIZATIONAL  GOAL  #2:  To  develop  a  cadre  of  highly  qualified  individuals 
well-suited  to  occupations  integral  to  the  future  National  Airspace 
System 


Objective  3:  To  improve  the  technical  competence  of  the  FAA  workforce 

Objective  4:  To  develop  within  the  FAA  workforce  an  increased  acceptance 

of  technological  change 

Objective  5:  To  recruit  individuals  with  managerial  potential 

Objective  6:  To  attract  individuals  with  greater  awareness  of  and  skills 

in  maintaining  positive  human  relations 


ORGANIZATIONAL  GOAL  #3:  To  promote  socially  responsible  personnel  policies  and 
practices 

Objective  7:  To  increase  female  and  minority  representation  within  the 
FAA 

Objective  8:  To  improve  employee  perceptions  of  the  FAA 
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